D13a_Hadoop Swimlanes - Ready! on Tez.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f4044-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f41c0-495a-11e8-ace7-0242ac130002",
     "previous": null
    }
   },
   "source": [
    "# About: Hadoop Swimlanes - Ready! on Tez\n",
    "\n",
    "----\n",
    "\n",
    "Tezがインストールされた環境にtez-toolsをインストールし、Swimlanesを使用できるようにします。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f41c0-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f430a-495a-11e8-ace7-0242ac130002",
     "previous": "a75f4044-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "## *Operation Note*\n",
    "\n",
    "*This is a cell for your own recording. ここに経緯を記述*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f430a-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f444a-495a-11e8-ace7-0242ac130002",
     "previous": "a75f41c0-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "# 操作対象クラスタの設定\n",
    "\n",
    "起動したい対象のクラスタ名を設定する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f444a-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f4580-495a-11e8-ace7-0242ac130002",
     "previous": "a75f430a-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [],
   "source": [
    "target_group = 'hadoop_all_testcluster'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f4580-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f46c0-495a-11e8-ace7-0242ac130002",
     "previous": "a75f444a-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "対象クラスタにAnsibleでpingできることを確認する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f46c0-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f47ec-495a-11e8-ace7-0242ac130002",
     "previous": "a75f4580-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32mXXX.XXX.XXX.72 | SUCCESS => {\n",
      "    \"changed\": false, \n",
      "    \"ping\": \"pong\"\n",
      "}\u001b[0m\n",
      "\u001b[0;32mXXX.XXX.XXX.73 | SUCCESS => {\n",
      "    \"changed\": false, \n",
      "    \"ping\": \"pong\"\n",
      "}\u001b[0m\n",
      "\u001b[0;32mXXX.XXX.XXX.112 | SUCCESS => {\n",
      "    \"changed\": false, \n",
      "    \"ping\": \"pong\"\n",
      "}\u001b[0m\n",
      "\u001b[0;32mXXX.XXX.XXX.70 | SUCCESS => {\n",
      "    \"changed\": false, \n",
      "    \"ping\": \"pong\"\n",
      "}\u001b[0m\n",
      "\u001b[0;32mXXX.XXX.XXX.71 | SUCCESS => {\n",
      "    \"changed\": false, \n",
      "    \"ping\": \"pong\"\n",
      "}\u001b[0m\n",
      "\u001b[0;32mXXX.XXX.XXX.113 | SUCCESS => {\n",
      "    \"changed\": false, \n",
      "    \"ping\": \"pong\"\n",
      "}\u001b[0m\n",
      "\u001b[0;32mXXX.XXX.XXX.114 | SUCCESS => {\n",
      "    \"changed\": false, \n",
      "    \"ping\": \"pong\"\n",
      "}\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!ansible -m ping {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f47ec-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f52e6-495a-11e8-ace7-0242ac130002",
     "previous": "a75f46c0-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "インストール対象は、 `hadoop_swimlane` というグループに所属しているものとする。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f52e6-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f5444-495a-11e8-ace7-0242ac130002",
     "previous": "a75f47ec-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32mXXX.XXX.XXX.72 | SUCCESS => {\r\n",
      "    \"changed\": false, \r\n",
      "    \"ping\": \"pong\"\r\n",
      "}\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible -m ping hadoop_swimlane -l {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f5444-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f5584-495a-11e8-ace7-0242ac130002",
     "previous": "a75f52e6-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "# 設定の定義\n",
    "\n",
    "インストール対象パスなど、設定を定義する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "a75f5584-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f56c4-495a-11e8-ace7-0242ac130002",
     "previous": "a75f5444-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [],
   "source": [
    "swimlane_install_path = '/usr/local/lib'\n",
    "swimlane_tmp_path = '/tmp'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f56c4-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f5958-495a-11e8-ace7-0242ac130002",
     "previous": "a75f5584-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "インストールするtez-toolsのバージョンを定義する。gitからcloneしてくるためのtagも定義しておく。\n",
    "\n",
    "https://github.com/apache/tez"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "a75f5958-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f5c64-495a-11e8-ace7-0242ac130002",
     "previous": "a75f56c4-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [],
   "source": [
    "swimlane_tez_version = '0.8.4'\n",
    "swimlane_tez_repo_tag = 'rel/release-' + swimlane_tez_version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f5c64-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f6060-495a-11e8-ace7-0242ac130002",
     "previous": "a75f5958-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "Playbookなど定義するための一時ディレクトリを定義しておく。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f6060-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f6290-495a-11e8-ace7-0242ac130002",
     "previous": "a75f5c64-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/tmp/tmpSixtFf'"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "import tempfile\n",
    "\n",
    "work_dir = tempfile.mkdtemp()\n",
    "work_dir"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f6290-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f63da-495a-11e8-ace7-0242ac130002",
     "previous": "a75f6060-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "# tez-toolsのインストール\n",
    "\n",
    "Gitからcloneしてくる・・・まずは対象ホストにgitをインストールする。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f63da-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f6588-495a-11e8-ace7-0242ac130002",
     "previous": "a75f6290-495a-11e8-ace7-0242ac130002"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;33mXXX.XXX.XXX.72 | SUCCESS => {\r\n",
      "    \"changed\": true, \r\n",
      "    \"msg\": \"\", \r\n",
      "    \"rc\": 0, \r\n",
      "    \"results\": [\r\n",
      "        \"Loaded plugins: fastestmirror\\nSetting up Install Process\\nLoading mirror speeds from cached hostfile\\n * base: ftp.iij.ad.jp\\n * extras: ftp.iij.ad.jp\\n * updates: ftp.iij.ad.jp\\nResolving Dependencies\\n--> Running transaction check\\n---> Package git.x86_64 0:1.7.1-4.el6_7.1 will be installed\\n--> Processing Dependency: perl-Git = 1.7.1-4.el6_7.1 for package: git-1.7.1-4.el6_7.1.x86_64\\n--> Processing Dependency: rsync for package: git-1.7.1-4.el6_7.1.x86_64\\n--> Processing Dependency: perl(Git) for package: git-1.7.1-4.el6_7.1.x86_64\\n--> Processing Dependency: perl(Error) for package: git-1.7.1-4.el6_7.1.x86_64\\n--> Running transaction check\\n---> Package perl-Error.noarch 1:0.17015-4.el6 will be installed\\n---> Package perl-Git.noarch 0:1.7.1-4.el6_7.1 will be installed\\n---> Package rsync.x86_64 0:3.0.6-12.el6 will be installed\\n--> Finished Dependency Resolution\\n\\nDependencies Resolved\\n\\n================================================================================\\n Package            Arch           Version                   Repository    Size\\n================================================================================\\nInstalling:\\n git                x86_64         1.7.1-4.el6_7.1           base         4.6 M\\nInstalling for dependencies:\\n perl-Error         noarch         1:0.17015-4.el6           base          29 k\\n perl-Git           noarch         1.7.1-4.el6_7.1           base          28 k\\n rsync              x86_64         3.0.6-12.el6              base         335 k\\n\\nTransaction Summary\\n================================================================================\\nInstall       4 Package(s)\\n\\nTotal download size: 5.0 M\\nInstalled size: 15 M\\nDownloading Packages:\\n--------------------------------------------------------------------------------\\nTotal                                            18 MB/s | 5.0 MB     00:00     \\nRunning rpm_check_debug\\nRunning Transaction Test\\nTransaction Test Succeeded\\nRunning Transaction\\n\\r  Installing : 1:perl-Error-0.17015-4.el6.noarch                            1/4 \\n\\r  Installing : rsync-3.0.6-12.el6.x86_64                                    2/4 \\n\\r  Installing : git-1.7.1-4.el6_7.1.x86_64                                   3/4 \\n\\r  Installing : perl-Git-1.7.1-4.el6_7.1.noarch                              4/4 \\n\\r  Verifying  : perl-Git-1.7.1-4.el6_7.1.noarch                              1/4 \\n\\r  Verifying  : rsync-3.0.6-12.el6.x86_64                                    2/4 \\n\\r  Verifying  : git-1.7.1-4.el6_7.1.x86_64                                   3/4 \\n\\r  Verifying  : 1:perl-Error-0.17015-4.el6.noarch                            4/4 \\n\\nInstalled:\\n  git.x86_64 0:1.7.1-4.el6_7.1                                                  \\n\\nDependency Installed:\\n  perl-Error.noarch 1:0.17015-4.el6      perl-Git.noarch 0:1.7.1-4.el6_7.1     \\n  rsync.x86_64 0:3.0.6-12.el6           \\n\\nComplete!\\n\"\r\n",
      "    ]\r\n",
      "}\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible -b -m yum -a 'name=git state=latest' hadoop_swimlane -l {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f6588-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f6ea2-495a-11e8-ace7-0242ac130002",
     "previous": "a75f63da-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "リポジトリからcloneしてくる。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f6ea2-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f7906-495a-11e8-ace7-0242ac130002",
     "previous": "a75f6588-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;33mXXX.XXX.XXX.72 | SUCCESS => {\r\n",
      "    \"after\": \"ef70407682918c022dffea86d6fa0571ccebcd8b\", \r\n",
      "    \"before\": null, \r\n",
      "    \"changed\": true\r\n",
      "}\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible -b -m git -a 'repo=https://github.com/apache/tez.git dest={ swimlane_install_path }/apache-tez-{ swimlane_tez_version }-src version={ swimlane_tez_repo_tag }' \\\n",
    "         hadoop_swimlane -l {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f7906-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f7b72-495a-11e8-ace7-0242ac130002",
     "previous": "a75f6ea2-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "yarnユーザで実行することを想定して、ディレクトリのOwner, Permissionを調整しておく。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f7b72-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f7db6-495a-11e8-ace7-0242ac130002",
     "previous": "a75f7906-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;33mXXX.XXX.XXX.72 | SUCCESS => {\r\n",
      "    \"changed\": true, \r\n",
      "    \"gid\": 496, \r\n",
      "    \"group\": \"hadoop\", \r\n",
      "    \"mode\": \"0777\", \r\n",
      "    \"owner\": \"yarn\", \r\n",
      "    \"path\": \"/usr/local/lib/apache-tez-0.8.4-src/tez-tools/swimlanes\", \r\n",
      "    \"size\": 4096, \r\n",
      "    \"state\": \"directory\", \r\n",
      "    \"uid\": 496\r\n",
      "}\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible -b -m file -a 'path={ swimlane_install_path }/apache-tez-{ swimlane_tez_version }-src/tez-tools/swimlanes state=directory mode=777 owner=yarn group=hadoop' \\\n",
    "         hadoop_swimlane -l {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f7db6-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f7fdc-495a-11e8-ace7-0242ac130002",
     "previous": "a75f7b72-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "`yarn-swimlanes.sh` が作成されていることを確認する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f7fdc-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f81d0-495a-11e8-ace7-0242ac130002",
     "previous": "a75f7db6-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32mXXX.XXX.XXX.72 | SUCCESS | rc=0 >>\r\n",
      "-rw-r--r-- 1 root root 980 Aug 19 10:21 /usr/local/lib/apache-tez-0.8.4-src/tez-tools/swimlanes/yarn-swimlanes.sh\r\n",
      "\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible -a 'ls -la { swimlane_install_path }/apache-tez-{ swimlane_tez_version }-src/tez-tools/swimlanes/yarn-swimlanes.sh' \\\n",
    "         hadoop_swimlane -l {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f81d0-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f83b0-495a-11e8-ace7-0242ac130002",
     "previous": "a75f7fdc-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "# group_varsパラメータの更新\n",
    "\n",
    "このクラスタのgroup_varsにSwimlane関連のパラメータを追加しておく。いったん現在のgroup_varsをバックアップしておく。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f83b0-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f859a-495a-11e8-ace7-0242ac130002",
     "previous": "a75f81d0-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [],
   "source": [
    "!cp group_vars/{target_group} {work_dir}/group_vars-old"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f859a-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f8798-495a-11e8-ace7-0242ac130002",
     "previous": "a75f83b0-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "[group_vars](../tree/group_vars) ディレクトリ中の `group_vars/hadoop_all_{{クラスタ名}}` に以下を追記する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f8798-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f890a-495a-11e8-ace7-0242ac130002",
     "previous": "a75f859a-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Swimlane\n",
      "swimlane_install_path: /usr/local/lib\n",
      "swimlane_tez_version: 0.8.4\n",
      "swimlane_tmp_path: /tmp\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import yaml\n",
    "swimlane_vars = {'swimlane_install_path': swimlane_install_path, 'swimlane_tmp_path': swimlane_tmp_path,\n",
    "                 'swimlane_tez_version': swimlane_tez_version}\n",
    "print('# Swimlane\\n' + yaml.dump(swimlane_vars, default_flow_style=False))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f890a-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f8a54-495a-11e8-ace7-0242ac130002",
     "previous": "a75f8798-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- /tmp/tmpBi2ROi/group_vars-old\t2016-08-19 10:27:23.854033715 +0900\r\n",
      "+++ group_vars/hadoop_all_testcluster\t2016-08-19 10:32:01.916381623 +0900\r\n",
      "@@ -47,8 +47,6 @@\r\n",
      " # Java\r\n",
      " jdk7_downloadurl: 'http://xxx.xxx.nii.ac.jp:YYYY/v1/AUTH_XXXXXXXXXXXXXXXXXXXXXXXX/HDP-2.x/archive'\r\n",
      " jdk8_downloadurl: 'http://xxx.xxx.nii.ac.jp:YYYY/v1/AUTH_XXXXXXXXXXXXXXXXXXXXXXXX/HDP-2.x/archive'\r\n",
      "-# Swimlane\r\n",
      "-swimlane_tez_version: '0.8.4'\r\n",
      " \r\n",
      " # HDFS\r\n",
      " dfs_namenode_name_dirs:\r\n",
      "@@ -80,3 +78,9 @@\r\n",
      " # Spark\r\n",
      " spark_download_url: 'http://xxx.xxx.nii.ac.jp:YYYY/v1/AUTH_XXXXXXXXXXXXXXXXXXXXXXXX/HDP-2.x/archive'\r\n",
      " spark_history_server_host: 'testvm003'\r\n",
      "+\r\n",
      "+# Swimlane\r\n",
      "+swimlane_install_path: /usr/local/lib\r\n",
      "+swimlane_tez_version: 0.8.4\r\n",
      "+swimlane_tmp_path: /tmp\r\n",
      "+\r\n"
     ]
    }
   ],
   "source": [
    "!diff -u {work_dir}/group_vars-old group_vars/{target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f8a54-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f8ba8-495a-11e8-ace7-0242ac130002",
     "previous": "a75f890a-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "# 動作確認\n",
    "\n",
    "YARNでサンプルジョブを実行してみて、Swimlaneの出力が得られるかを確認する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f8ba8-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f8ce8-495a-11e8-ace7-0242ac130002",
     "previous": "a75f8a54-495a-11e8-ace7-0242ac130002"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XXX.XXX.XXX.72 | SUCCESS | rc=0 >>\r\n",
      "Number of Maps  = 10\r\n",
      "Samples per Map = 1000\r\n",
      "Wrote input for Map #0\r\n",
      "Wrote input for Map #1\r\n",
      "Wrote input for Map #2\r\n",
      "Wrote input for Map #3\r\n",
      "Wrote input for Map #4\r\n",
      "Wrote input for Map #5\r\n",
      "Wrote input for Map #6\r\n",
      "Wrote input for Map #7\r\n",
      "Wrote input for Map #8\r\n",
      "Wrote input for Map #9\r\n",
      "Starting Job\r\n",
      "Job Finished in 25.773 seconds\r\n",
      "Estimated value of Pi is 3.1408000000000000000016/08/19 10:49:35 INFO impl.TimelineClientImpl: Timeline service address: http://testvm003:8188/ws/v1/timeline/\r\n",
      "16/08/19 10:49:36 INFO input.FileInputFormat: Total input paths to process : 10\r\n",
      "16/08/19 10:49:36 INFO mapreduce.JobSubmitter: number of splits:10\r\n",
      "16/08/19 10:49:37 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1471559811619_0004\r\n",
      "16/08/19 10:49:37 INFO client.YARNRunner: Number of stages: 2\r\n",
      "16/08/19 10:49:37 INFO client.TezClient: Tez Client Version: [ component=tez-api, version=XXX.XXX.XXX.2.4.2.0-258, revision=fa554fdce4e3495e09a310e0a32bb34ccc5946ad, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160425-0638 ]\r\n",
      "16/08/19 10:49:38 INFO impl.TimelineClientImpl: Timeline service address: http://testvm003:8188/ws/v1/timeline/\r\n",
      "16/08/19 10:49:38 INFO client.TezClient: Submitting DAG application with id: application_1471559811619_0004\r\n",
      "16/08/19 10:49:38 INFO client.TezClientUtils: Using tez.lib.uris value from configuration: hdfs://hdfs-cluster//apps/tez/tez.tar.gz\r\n",
      "16/08/19 10:49:38 INFO client.TezClient: Tez system stage directory hdfs://hdfs-cluster/tmp/hadoop-yarn/staging/ansible/.staging/job_1471559811619_0004/.tez/application_1471559811619_0004 doesn't exist and is created\r\n",
      "16/08/19 10:49:38 INFO client.TezClient: Submitting DAG to YARN, applicationId=application_1471559811619_0004, dagName=QuasiMonteCarlo\r\n",
      "16/08/19 10:49:38 INFO impl.YarnClientImpl: Submitted application application_1471559811619_0004\r\n",
      "16/08/19 10:49:38 INFO client.TezClient: The url to track the Tez AM: http://testvm001:8088/proxy/application_1471559811619_0004/\r\n",
      "16/08/19 10:49:39 INFO impl.TimelineClientImpl: Timeline service address: http://testvm003:8188/ws/v1/timeline/\r\n",
      "16/08/19 10:49:39 INFO impl.TimelineClientImpl: Timeline service address: http://testvm003:8188/ws/v1/timeline/\r\n",
      "16/08/19 10:49:39 INFO mapreduce.Job: The url to track the job: http://testvm001:8088/proxy/application_1471559811619_0004/\r\n",
      "16/08/19 10:49:39 INFO mapreduce.Job: Running job: job_1471559811619_0004\r\n",
      "16/08/19 10:49:46 INFO mapreduce.Job: Job job_1471559811619_0004 running in uber mode : false\r\n",
      "16/08/19 10:49:46 INFO mapreduce.Job:  map 0% reduce 0%\r\n",
      "16/08/19 10:49:54 INFO mapreduce.Job:  map 30% reduce 0%\r\n",
      "16/08/19 10:49:55 INFO mapreduce.Job:  map 70% reduce 0%\r\n",
      "16/08/19 10:49:59 INFO mapreduce.Job:  map 90% reduce 0%\r\n",
      "16/08/19 10:50:00 INFO mapreduce.Job:  map 100% reduce 0%\r\n",
      "16/08/19 10:50:00 INFO mapreduce.Job:  map 100% reduce 100%\r\n",
      "16/08/19 10:50:00 INFO mapreduce.Job: Job job_1471559811619_0004 completed successfully\r\n",
      "16/08/19 10:50:00 INFO mapreduce.Job: Counters: 0\r\n"
     ]
    }
   ],
   "source": [
    "yarn_result = os.path.join(work_dir, 'yarn-result')\n",
    "!ansible -a \"yarn jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples.jar pi 10 1000\" -l { target_group } hadoop_client \\\n",
    "         | tee {yarn_result}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f8ce8-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f8ea0-495a-11e8-ace7-0242ac130002",
     "previous": "a75f8ba8-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "Swimlane実行用Playbookを準備する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f8ea0-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f908a-495a-11e8-ace7-0242ac130002",
     "previous": "a75f8ce8-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing /tmp/tmpSixtFf/swimlane.yml\n"
     ]
    }
   ],
   "source": [
    "%%writefile {work_dir}/swimlane.yml\n",
    "- hosts: hadoop_swimlane\n",
    "  tasks:\n",
    "    - name: delete_swimlane_old_output\n",
    "      become: yes\n",
    "      file: path={{ swimlane_install_path }}/apache-tez-{{ swimlane_tez_version }}-src/tez-tools/swimlanes/{{ yarn_application_id }}.svg state=absent\n",
    "  \n",
    "    - name: create_swimlane_analyze\n",
    "      shell: bash yarn-swimlanes.sh {{ yarn_application_id }}\n",
    "      args:\n",
    "        chdir: \"{{ swimlane_install_path }}/apache-tez-{{ swimlane_tez_version }}-src/tez-tools/swimlanes/\"\n",
    "  \n",
    "    - name: download_svg\n",
    "      fetch: src={{ swimlane_install_path }}/apache-tez-{{ swimlane_tez_version }}-src/tez-tools/swimlanes/{{ yarn_application_id }}.svg dest={{ local_svg_path }} flat=yes\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f908a-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f9260-495a-11e8-ace7-0242ac130002",
     "previous": "a75f8ea0-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "先に実行したpiの application Idを得ておく。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f9260-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f93aa-495a-11e8-ace7-0242ac130002",
     "previous": "a75f908a-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'application_1471559811619_0004'"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yarn_application_id = None\n",
    "with open(yarn_result, 'r') as f:\n",
    "    for line in f:\n",
    "        if 'Submitting DAG application with id' in line:\n",
    "            yarn_application_id = line.split()[-1]\n",
    "assert(yarn_application_id is not None)\n",
    "yarn_application_id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f93aa-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f94e0-495a-11e8-ace7-0242ac130002",
     "previous": "a75f9260-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "swimlaneのスクリプトを実行し、SVGファイルをダウンロードする。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f94e0-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f9620-495a-11e8-ace7-0242ac130002",
     "previous": "a75f93aa-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "PLAY [hadoop_swimlane] *********************************************************\n",
      "\n",
      "TASK [setup] *******************************************************************\n",
      "\u001b[0;32mok: [XXX.XXX.XXX.72]\u001b[0m\n",
      "\n",
      "TASK [delete_swimlane_old_output] **********************************************\n",
      "\u001b[0;32mok: [XXX.XXX.XXX.72]\u001b[0m\n",
      "\n",
      "TASK [create_swimlane_analyze] *************************************************\n",
      "\u001b[0;33mchanged: [XXX.XXX.XXX.72]\u001b[0m\n",
      "\n",
      "TASK [download_svg] ************************************************************\n",
      "\u001b[0;33mchanged: [XXX.XXX.XXX.72]\u001b[0m\n",
      "\n",
      "PLAY RECAP *********************************************************************\n",
      "\u001b[0;33mXXX.XXX.XXX.72\u001b[0m               : \u001b[0;32mok\u001b[0m\u001b[0;32m=\u001b[0m\u001b[0;32m4\u001b[0m    \u001b[0;33mchanged\u001b[0m\u001b[0;33m=\u001b[0m\u001b[0;33m2\u001b[0m    unreachable=0    failed=0   \n",
      "\n"
     ]
    }
   ],
   "source": [
    "local_svg_path = os.path.join(work_dir, 'result.svg')\n",
    "extra_vars = ['-e yarn_application_id={}'.format(yarn_application_id), '-e local_svg_path={}'.format(local_svg_path)]\n",
    "!ansible-playbook {' '.join(extra_vars)} {work_dir}/swimlane.yml -l {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f9620-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f9756-495a-11e8-ace7-0242ac130002",
     "previous": "a75f94e0-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "ダウンロードしたSVGファイルをレンダリングする。\n",
    "\n",
    "(以下にSwimlanesの出力画像が表示されない場合、JupyterのFileメニューからTrust Notebookすることで出力画像が表示されるようになります。)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "lc_cell_meme": {
     "current": "a75f9756-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f988c-495a-11e8-ace7-0242ac130002",
     "previous": "a75f9620-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"328\" version=\"1.1\" width=\"2711\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "\t\t<script type=\"text/ecmascript\" xlink:href=\"http://code.jquery.com/jquery-2.1.1.min.js\"/>\n",
       "\t\t<text style=\"font-size: 32px; text-anchor: middle\" transform=\"\" x=\"1127\" y=\"32\">appattempt_1471559811619_0004_000001</text><text style=\"text-anchor:end; font-size: 16px;\" transform=\"\" x=\"84\" y=\"96\">Container ID</text><text style=\"text-anchor:end; font-size: 16px;\" transform=\"\" x=\"96\" y=\"152\">container_e01_1471559811619_0004_01_000003</text><line style=\"stroke: #ccc\" x1=\"100\" x2=\"2355\" y1=\"152\" y2=\"152\"/><text style=\"text-anchor:end; font-size: 16px;\" transform=\"\" x=\"96\" y=\"176\">container_e01_1471559811619_0004_01_000004</text><line style=\"stroke: #ccc\" x1=\"100\" x2=\"2355\" y1=\"176\" y2=\"176\"/><text style=\"text-anchor:end; font-size: 16px;\" transform=\"\" x=\"96\" y=\"200\">container_e01_1471559811619_0004_01_000002</text><line style=\"stroke: #ccc\" x1=\"100\" x2=\"2355\" y1=\"200\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"100\" y=\"116\">0.00 s</text><line style=\"stroke: #ddd\" x1=\"100\" x2=\"100\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"870\" y=\"116\">5.00 s</text><line style=\"stroke: #ddd\" x1=\"870\" x2=\"870\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1640\" y=\"116\">10.00 s</text><line style=\"stroke: #ddd\" x1=\"1640\" x2=\"1640\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"240\" y=\"116\">0.00 s</text><line style=\"stroke: #ddd\" x1=\"240\" x2=\"240\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1010\" y=\"116\">6.00 s</text><line style=\"stroke: #ddd\" x1=\"1010\" x2=\"1010\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1780\" y=\"116\">11.00 s</text><line style=\"stroke: #ddd\" x1=\"1780\" x2=\"1780\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"380\" y=\"116\">1.00 s</text><line style=\"stroke: #ddd\" x1=\"380\" x2=\"380\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1150\" y=\"116\">7.00 s</text><line style=\"stroke: #ddd\" x1=\"1150\" x2=\"1150\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1920\" y=\"116\">12.00 s</text><line style=\"stroke: #ddd\" x1=\"1920\" x2=\"1920\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"520\" y=\"116\">2.00 s</text><line style=\"stroke: #ddd\" x1=\"520\" x2=\"520\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1290\" y=\"116\">8.00 s</text><line style=\"stroke: #ddd\" x1=\"1290\" x2=\"1290\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"2060\" y=\"116\">13.00 s</text><line style=\"stroke: #ddd\" x1=\"2060\" x2=\"2060\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"660\" y=\"116\">3.00 s</text><line style=\"stroke: #ddd\" x1=\"660\" x2=\"660\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1430\" y=\"116\">9.00 s</text><line style=\"stroke: #ddd\" x1=\"1430\" x2=\"1430\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"2200\" y=\"116\">14.00 s</text><line style=\"stroke: #ddd\" x1=\"2200\" x2=\"2200\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"800\" y=\"116\">4.00 s</text><line style=\"stroke: #ddd\" x1=\"800\" x2=\"800\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1570\" y=\"116\">10.00 s</text><line style=\"stroke: #ddd\" x1=\"1570\" x2=\"1570\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"2340\" y=\"116\">15.00 s</text><line style=\"stroke: #ddd\" x1=\"2340\" x2=\"2340\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"170\" y=\"116\">0.00 s</text><line style=\"stroke: #ddd\" x1=\"170\" x2=\"170\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"940\" y=\"116\">5.00 s</text><line style=\"stroke: #ddd\" x1=\"940\" x2=\"940\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1710\" y=\"116\">11.00 s</text><line style=\"stroke: #ddd\" x1=\"1710\" x2=\"1710\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"2355\" y=\"116\">15.00 s</text><line style=\"stroke: #ddd\" x1=\"2355\" x2=\"2355\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"310\" y=\"116\">1.00 s</text><line style=\"stroke: #ddd\" x1=\"310\" x2=\"310\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1080\" y=\"116\">6.00 s</text><line style=\"stroke: #ddd\" x1=\"1080\" x2=\"1080\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1850\" y=\"116\">12.00 s</text><line style=\"stroke: #ddd\" x1=\"1850\" x2=\"1850\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"450\" y=\"116\">2.00 s</text><line style=\"stroke: #ddd\" x1=\"450\" x2=\"450\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1220\" y=\"116\">7.00 s</text><line style=\"stroke: #ddd\" x1=\"1220\" x2=\"1220\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1990\" y=\"116\">13.00 s</text><line style=\"stroke: #ddd\" x1=\"1990\" x2=\"1990\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"590\" y=\"116\">3.00 s</text><line style=\"stroke: #ddd\" x1=\"590\" x2=\"590\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1360\" y=\"116\">8.00 s</text><line style=\"stroke: #ddd\" x1=\"1360\" x2=\"1360\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"2130\" y=\"116\">14.00 s</text><line style=\"stroke: #ddd\" x1=\"2130\" x2=\"2130\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"730\" y=\"116\">4.00 s</text><line style=\"stroke: #ddd\" x1=\"730\" x2=\"730\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"1500\" y=\"116\">9.00 s</text><line style=\"stroke: #ddd\" x1=\"1500\" x2=\"1500\" y1=\"116\" y2=\"200\"/><text style=\"text-anchor: middle; font-size: 12px\" transform=\"\" x=\"2270\" y=\"116\">15.00 s</text><line style=\"stroke: #ddd\" x1=\"2270\" x2=\"2270\" y1=\"116\" y2=\"200\"/><line style=\"stroke: #000\" x1=\"100\" x2=\"2355\" y1=\"128\" y2=\"128\"/><line style=\"stroke: #000\" x1=\"100\" x2=\"2355\" y1=\"200\" y2=\"200\"/><line style=\"stroke: #000\" x1=\"100\" x2=\"100\" y1=\"128\" y2=\"200\"/><line style=\"stroke: #000\" x1=\"2355\" x2=\"2355\" y1=\"128\" y2=\"200\"/><line style=\"stroke: green\" x1=\"561\" x2=\"561\" y1=\"128\" y2=\"152\"/><line style=\"stroke: green\" x1=\"3109\" x2=\"3109\" y1=\"128\" y2=\"152\"/><rect height=\"24\" style=\"fill: #ccc; opacity: 0.3\" width=\"2548\" x=\"561\" y=\"128\"><title/></rect><line style=\"stroke: green\" x1=\"561\" x2=\"561\" y1=\"176\" y2=\"200\"/><line style=\"stroke: green\" x1=\"2906\" x2=\"2906\" y1=\"176\" y2=\"200\"/><rect height=\"24\" style=\"fill: #ccc; opacity: 0.3\" width=\"2345\" x=\"561\" y=\"176\"><title/></rect><line style=\"stroke: green\" x1=\"561\" x2=\"561\" y1=\"152\" y2=\"176\"/><line style=\"stroke: green\" x1=\"3109\" x2=\"3109\" y1=\"152\" y2=\"176\"/><rect height=\"24\" style=\"fill: #ccc; opacity: 0.3\" width=\"2548\" x=\"561\" y=\"152\"><title/></rect><line stroke-dasharray=\"8,4\" style=\"stroke: black;\" x1=\"227\" x2=\"227\" y1=\"104\" y2=\"200\"/><line stroke-dasharray=\"8,4\" style=\"stroke: black;\" x1=\"2383\" x2=\"2383\" y1=\"104\" y2=\"200\"/><line style=\"stroke: black\" x1=\"227\" x2=\"2383\" y1=\"104\" y2=\"104\"/><text style=\"text-anchor: middle; font-size: 12px;\" transform=\"\" x=\"1305\" y=\"96\">dag_1471559811619_0004_1 (15.1 s)</text><rect height=\"22\" style=\"fill: #E2F2D8; stroke: #ccc;\" width=\"423\" x=\"921\" y=\"153\"><title>attempt_1471559811619_0004_1_00_000002_0</title></rect><text style=\"text-anchor: middle; font-size: 9px;\" transform=\"\" x=\"1132\" y=\"163\">initialmap (00002_0)</text><rect height=\"22\" style=\"fill: #E2F2D8; stroke: #ccc;\" width=\"413\" x=\"935\" y=\"177\"><title>attempt_1471559811619_0004_1_00_000000_0</title></rect><text style=\"text-anchor: middle; font-size: 9px;\" transform=\"\" x=\"1141\" y=\"187\">initialmap (00000_0)</text><rect height=\"22\" style=\"fill: #E2F2D8; stroke: #ccc;\" width=\"25\" x=\"1360\" y=\"153\"><title>attempt_1471559811619_0004_1_00_000003_0</title></rect><text style=\"text-anchor: middle; font-size: 9px;\" transform=\"\" x=\"1372\" y=\"163\">initialmap</text><rect height=\"22\" style=\"fill: #E2F2D8; stroke: #ccc;\" width=\"390\" x=\"1012\" y=\"129\"><title>attempt_1471559811619_0004_1_00_000001_0</title></rect><text style=\"text-anchor: middle; font-size: 9px;\" transform=\"\" x=\"1207\" y=\"139\">initialmap (00001_0)</text><rect height=\"22\" style=\"fill: #E2F2D8; stroke: #ccc;\" width=\"19\" x=\"1418\" y=\"129\"><title>attempt_1471559811619_0004_1_00_000007_0</title></rect><text style=\"text-anchor: middle; font-size: 9px;\" transform=\"\" x=\"1427\" y=\"139\">initialmap</text><rect height=\"22\" style=\"fill: #E2F2D8; stroke: #ccc;\" width=\"20\" x=\"1452\" y=\"129\"><title>attempt_1471559811619_0004_1_00_000008_0</title></rect><text style=\"text-anchor: middle; font-size: 9px;\" transform=\"\" x=\"1462\" y=\"139\">initialmap</text><rect height=\"22\" style=\"fill: #E2F2D8; stroke: #ccc;\" width=\"77\" x=\"1400\" y=\"153\"><title>attempt_1471559811619_0004_1_00_000005_0</title></rect><text style=\"text-anchor: middle; font-size: 9px;\" transform=\"\" x=\"1438\" y=\"163\">initialmap (00005_0)</text><rect height=\"22\" style=\"fill: #E2F2D8; stroke: #ccc;\" width=\"554\" x=\"1488\" y=\"129\"><title>attempt_1471559811619_0004_1_00_000009_0</title></rect><text style=\"text-anchor: middle; font-size: 9px;\" transform=\"\" x=\"1765\" y=\"139\">initialmap (00009_0)</text><rect height=\"22\" style=\"fill: #E2F2D8; stroke: #ccc;\" width=\"684\" x=\"1364\" y=\"177\"><title>attempt_1471559811619_0004_1_00_000004_0</title></rect><text style=\"text-anchor: middle; font-size: 9px;\" transform=\"\" x=\"1706\" y=\"187\">initialmap (00004_0)</text><rect height=\"22\" style=\"fill: #E2F2D8; stroke: #ccc;\" width=\"745\" x=\"1491\" y=\"153\"><title>attempt_1471559811619_0004_1_00_000006_0</title></rect><text style=\"text-anchor: middle; font-size: 9px;\" transform=\"\" x=\"1863\" y=\"163\">initialmap (00006_0)</text><rect height=\"22\" style=\"fill: #62C2A2; stroke: #ccc;\" width=\"298\" x=\"2057\" y=\"129\"><title>attempt_1471559811619_0004_1_01_000000_0</title></rect><text style=\"text-anchor: middle; font-size: 9px;\" transform=\"\" x=\"2206\" y=\"139\">finalreduce (00000_0)</text></svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import SVG\n",
    "SVG(filename=local_svg_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "a75f988c-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f99c2-495a-11e8-ace7-0242ac130002",
     "previous": "a75f9756-495a-11e8-ace7-0242ac130002"
    }
   },
   "source": [
    "# 後始末\n",
    "\n",
    "一時ディレクトリを削除する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "a75f99c2-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": "a75f9af8-495a-11e8-ace7-0242ac130002",
     "previous": "a75f988c-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [],
   "source": [
    "!rm -fr {work_dir}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "a75f9af8-495a-11e8-ace7-0242ac130002",
     "history": [],
     "next": null,
     "previous": "a75f99c2-495a-11e8-ace7-0242ac130002"
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  },
  "lc_notebook_meme": {
   "current": "a75f3dc4-495a-11e8-ace7-0242ac130002",
   "history": [],
   "root_cells": [
    "a75f4044-495a-11e8-ace7-0242ac130002",
    "a75f41c0-495a-11e8-ace7-0242ac130002",
    "a75f430a-495a-11e8-ace7-0242ac130002",
    "a75f444a-495a-11e8-ace7-0242ac130002",
    "a75f4580-495a-11e8-ace7-0242ac130002",
    "a75f46c0-495a-11e8-ace7-0242ac130002",
    "a75f47ec-495a-11e8-ace7-0242ac130002",
    "a75f52e6-495a-11e8-ace7-0242ac130002",
    "a75f5444-495a-11e8-ace7-0242ac130002",
    "a75f5584-495a-11e8-ace7-0242ac130002",
    "a75f56c4-495a-11e8-ace7-0242ac130002",
    "a75f5958-495a-11e8-ace7-0242ac130002",
    "a75f5c64-495a-11e8-ace7-0242ac130002",
    "a75f6060-495a-11e8-ace7-0242ac130002",
    "a75f6290-495a-11e8-ace7-0242ac130002",
    "a75f63da-495a-11e8-ace7-0242ac130002",
    "a75f6588-495a-11e8-ace7-0242ac130002",
    "a75f6ea2-495a-11e8-ace7-0242ac130002",
    "a75f7906-495a-11e8-ace7-0242ac130002",
    "a75f7b72-495a-11e8-ace7-0242ac130002",
    "a75f7db6-495a-11e8-ace7-0242ac130002",
    "a75f7fdc-495a-11e8-ace7-0242ac130002",
    "a75f81d0-495a-11e8-ace7-0242ac130002",
    "a75f83b0-495a-11e8-ace7-0242ac130002",
    "a75f859a-495a-11e8-ace7-0242ac130002",
    "a75f8798-495a-11e8-ace7-0242ac130002",
    "a75f890a-495a-11e8-ace7-0242ac130002",
    "a75f8a54-495a-11e8-ace7-0242ac130002",
    "a75f8ba8-495a-11e8-ace7-0242ac130002",
    "a75f8ce8-495a-11e8-ace7-0242ac130002",
    "a75f8ea0-495a-11e8-ace7-0242ac130002",
    "a75f908a-495a-11e8-ace7-0242ac130002",
    "a75f9260-495a-11e8-ace7-0242ac130002",
    "a75f93aa-495a-11e8-ace7-0242ac130002",
    "a75f94e0-495a-11e8-ace7-0242ac130002",
    "a75f9620-495a-11e8-ace7-0242ac130002",
    "a75f9756-495a-11e8-ace7-0242ac130002",
    "a75f988c-495a-11e8-ace7-0242ac130002",
    "a75f99c2-495a-11e8-ace7-0242ac130002",
    "a75f9af8-495a-11e8-ace7-0242ac130002"
   ]
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}